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Abstract 


There is concern that online education may widen the achievement gap between 
students from different socioeconomic classes. The recent discussion of integrating 
massive open online courses (MOOCs) into formal higher education has added fuel to 
this debate. In this study, factors influencing enrollment and completion in a pre-college 
preparatory MOOC were explored. University of California at Irvine (UCI) students of 
all preparation levels, defined by math Scholastic Aptitude Test (SAT) score, were 
invited to take a Bio Prep MOOC to help them prepare for introductory biology. 
Students with math SAT below 550 were offered the explicit incentive of an early change 
to the biology major upon successful completion of the MOOC and two additional onsite 
courses. Our results demonstrate that, among course registrants, a higher percentage of 
UCI students (>60%) completed the course than non-UCI registrants from the general 
population (<9%). Female UCI students had a greater likelihood of enrolling in the 
MOOC, but were not different from male students in terms of performance. University 
students entering with low preparation outperformed students entering who already 
had the credentials to become biology majors. These findings suggest that MOOCs can 
reach students, even those entering college with less preparation, before they enter 
university and have the potential to prepare them for challenging science, technology, 
engineering, and mathematics (STEM) courses. 
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Introduction 


The demand for a diverse workforce of individuals with degrees in science, technology, 
engineering, and mathematics (STEM) has focused attention on how to increase the 
enrollment and retention of undergraduate STEM majors (National Science 
Foundation, 2004; National Science Board, 2006; Wilson et ah, 2012; National Science 
Board, 2012). Attrition in STEM majors is particularly high during the first two years of 
undergraduate education (Seymour & Hewitt, 1997 ; Shuman et ah, 1999; Chang, Cerna, 
Han, & Saenz, 2008). At many universities, large introductory STEM courses, often 
referred to as "gate-keepers," represent major barriers that discourage students from 
entering or persisting in STEM disciplines. Moreover, females (Felder, Felder, Mauney, 
Hamrin, & Dietz, 1995; Brainard & Carlin, 1997; Blickenstaff, 2005), first-generation 
college students (Engle & Tinto, 2008), students from low-income families (Lam, 
Srivatsan, Doverspike, Vesalo, & Mawasha, 2006), and under-represented minority 
students (Cole & Espinoza, 2008) are more likely to drop out from STEM majors. 

In recent years, with the number of college applications increasing (Clinedinst & 
Hawkins, 2014), there are growing numbers of low-income, under-represented 
minority, and first generation college students in entering classes. These students 
represent a tremendous resource for developing a diverse community of STEM 
graduates but they are also at high risk for dropping out since they are often 
academically under-prepared (Perna & Titus, 2005; Gumport, 2007; Breneman, Jr., & 
Hoxby, 1998). Providing these incoming students with access to low cost, high quality 
courses that help them transition to college and be successful in STEM majors is 
therefore of great importance. However, financial constraints have made it increasingly 
difficult for the brick-and-mortar campuses to provide transition face-to-face 
instruction to support these students (Clinedinst & Hawkins, 2014), especially in the 
summer before initial entry. 

The emergence of massive open online courses (MOOCs) potentially affords universities 
the opportunity to provide students with preparatory courses before they enter 
university, at relatively low cost. MOOCs are typically developed by university faculty 
and are free for anyone who has access to a computer and the internet. In general, the 
completion rate of stand-alone MOOCs is very low (about 5%) (Ho et ah, 2014). 
Enrollment surveys indicate that the students in the new massive courses are largely 
male and well-educated learners, rather than the underserved (Christensen et ah, 2013). 
A pilot study conducted in San Jose State University, which introduced MOOCs into the 
curriculum, reported that matriculated students performed better than non- 
matriculated students. However, the completion rate of the specifically targeted at-risk 
students was disappointingly low. It was reported that students’ effort was the strongest 
indicator of their success in the course (Firmin et al., 2014). In addition, students 
claiming a high intention to complete a MOOC in a pre-survey are much more likely to 
finish the course than others (Roller et al., 2013). This raises the question of whether it 
is possible to increase learners’ motivation, engagement, and success in MOOCs by 
providing external incentives. 
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The goal of this study was to develop a course that would help under-prepared students 
who had been accepted to the UCI gain skills and knowledge that would increase their 
probability of success in a large freshman STEM course, Bio 93. We chose to develop 
this as a MOOC to determine if this could be done in the context of a course that would 
also potentially benefit a broad group of individuals in the general population. We first 
introduce the rationale for providing this course, explain how it was organized, and 
describe the incentive offered to increase motivation for under-prepared students, and 
then analyze the extent to which the course achieved our goals. 

Research Questions 

The study addresses the following questions. 

1. How did UC Irvine (UCI) Bio 93 students perform in the MOOC compared to the 
general population students? 

2. Among UCI Bio 93 students, were underprepared students more likely to enroll in 
the MOOC given an explicit incentive? 

3. Among UCI Bio 93 students, were underprepared students more likely to complete 
the course? 

This paper focuses on student success within the Bio Prep MOOC; a second paper will 
focus on the performance by students in the subsequent face-to-face Bio 93 course. 


Data and Methods 


Pre-College Biology MOOC Design 

At UCI, Bio 93 is the first-quarter, introductory biology course that is required for all 
students majoring in biology or health careers. Author Diane K. O'Dowd has been 
teaching the class for 10 years, and author Adrienne E. Williams has worked with Diane 
K. O'Dowd to develop teaching techniques and measure student performance in the 
course for the past 8 years. Our analysis of data from the last three years indicates that 
Bio 93 is a major obstacle for under-represented minority students. The drop/fail rate 
for these groups in Bio 93 is 25-30%, much higher than the 15% overall class average. 
Further analysis of learner outcome data indicates that high school preparation is a 
critical factor in passing Bio 93. A high percentage of students who score below 550 on 
their quantitative Scholastic Aptitude Test (SAT) scores drop or fail Bio 93. Students 
must score 550 or above to be accepted as a biology major; those below that score are 
redirected into other majors, most commonly undecided/undeclared.We have 
investigated ways of creating preparatory courses and learning tools to meet the needs 
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of academically less prepared students, ideally before they enter the university. Online 
courses offer one way of potentially helping students who have not yet moved to Irvine, 
and free courses are potentially highly beneficial for this often low-income group. The 
Bio Prep MOOC was designed by the co-first author Adrienne E. Williams to be 
accessible to the general public, but was specifically aimed at providing knowledge and 
critical thinking that is important for success in Bio 93. It was offered on the Coursera 
MOOC platform. 

The MOOC consisted of three modules with the biological themes of membrane 
transport, protein synthesis and localization, and neurophysiology. Authors Adrienne E. 
Williams and Diane K. O’Dowd taught the course as two tracks. The Basics track was 
based on content videos and quizzes only. To complete the Scholars track, students 
needed to compose written responses to three to five short-answer questions in each of 
the three units of the course, and provide peer assessments (both numerical scores and 
comments) on short answers of at least three peers for each question. The Scholars 
Track also included three independent study projects, where students were required to 
apply what they have learned from the lectures to new situations. Students were also 
expected to participate in peer assessment of other students’ project reports to receive a 
grade for their own report. These additional writing and peer assessment activities were 
included in the Scholars track based on prior research indicating a correlation with 
academic success (Liang & Tsai, 2010). Students who successfully completed the 
Scholars track were awarded a Distinction certificate and those who completed the 
Basics track were awarded a Normal certificate. Students did not need to pre-select a 
track but were given the appropriate certificate based on the tasks they fulfilled. In 
summary, the Scholars track requires more self-regulated study for students than the 
Basics track. 

UCI Student Participation in the MOOC: Incentive for 
Underprepared Students 

Like all Coursera courses, Bio Prep was offered to the general public, and approximately 
37,000 students signed up. Beyond this general enrollment, we conducted targeted 
outreach for Bio Prep to incoming freshman students during summer Enrollment in 
early summer 2013, focusing on two groups in particular. Students accepted into the 
university as Biological Science majors were told about the course by the Biological 
Science counselors. They were encouraged to participate to help prepare them for a 
successful first quarter of Bio 93 but there was no explicit incentive. Another group of 
students applied to UCI with Biological Sciences as their first choice of major, but this 
choice was not granted because their math SAT scores were less than 550. Most of these 
students became Undecided majors. These students were also encouraged to participate 
in the Bio Prep MOOC by their Summer Enrollment counselors through the Undecided 
Student Affairs Office and they were additionally provided an explicit but no cost 
incentive to take the MOOC. Ordinarily, students below the 550 math SAT threshold 
must complete a full year of biology and chemistry courses successfully before entering 
the Biological Sciences major. However, students below this threshold were informed 
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that they could transfer to the major after only one quarter if they successfully 
completed the Bio Prep MOOC Scholars track and passed both Bio 93 and introductory 
chemistry during fall quarter. 

The MOOC was taught by Adrienne E. Williams and Diane K. O’Dowd in the four 
weeks immediately before fall quarter began at UCI (August 26 to September 23, 2013) 
to maximize the number of entering UCI students who could take the course. 

Subjects 

Subjects in the study included 27,487 MOOC students who had grade records in the Bio 
Prep MOOC, and the 1,695 UCI students who enrolled in Bio 93 in the fall 2013 quarter 
and for whom we had complete prior academic records. A subset of students were in 
both groups (n = 382). This subset was further divided into the “strong math UCI” 
students (n=226), who had SAT scores of 550 or higher, and the “weak math UCI” 
students (n=is6), who had SAT scores under 550. 

Dataset 

Data for the study came from three sources. The first is the SQL file extracted from 
Coursera database, which includes students’ assignment performance (e.g., quiz and 
peer assessment) and final performance in the Bio Prep MOOC (e.g., successfully 
completed the Scholars track and receiving the Distinction certificate). About 37,933 
students enrolled in the Bio Prep MOOC and 27,487 students had grade records. A total 
of 551 students earned the Distinction certificate and 1,971 students earned the Normal 
certificate. 

The second source of data comes from UCI’s Office of Institutional Research, which 
provides the demographic information and pre-college academic preparation record 
(i.e., SAT scores) of students who enrolled in the onsite Bio 93 course. 

Also, for UCI Bio 93 students who participated in the Bio Prep MOOC, we have the 
above data associated with their Coursera e-mail address and their UCI identification 
information. 

Student Information 

All MOOC participants (0=27,487) were coded as either UCI students (382) or non-UCI 
students (27,105). For all UCI Bio 93 students (n=i,695), including those in the MOOC, 
we gathered SAT math, reading, and writing scores. Literature indicates that online 
education may widen the academic performance gap between traditionally low- 
performing students and high-performing students (Xu & Jaggars, 2013). This study 
therefore examines how students’ previous academic performance influences their 
participation and performance in the MOOC. Students were also coded as “weak math” 
if their math SAT scores were below 550. As SAT math 550 is a threshold for the 
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university incentive policy, this variable reflects the effect of an explicit incentive policy 
on students’ participation and performance in the Bio Prep MOOC. 

Each UCI student was identified by gender and ethnicity. Prior studies show 
contradictory results in terms of which gender is more likely to take online courses and 
perform better in online learning. The analysis of University of Pennsylvania’s 36 
MOOCs indicates that there are significantly more males than females taking MOOCs 
(Christensen et al., 2013). Some research on distance education shows that there is no 
gender difference in learning outcomes in online learning while some research indicates 
that females perform significantly better than males (Xu & Jaggars, 2013). It has been 
argued that females are more motivated and better at communicating and at scheduling 
their online learning (McSporran & Young, 2001). Ethnicity was coded as under¬ 
represented minority (URM, i.e., Black/African-American, American Indian, and 
Hispanic) or non-URM. A number of studies report that the educational gap between 
URMs and non-URMs is wider in online courses than in face-to-face courses (for an 
overview, see Means, Bakia, and Murphy, 2014). 

Analyses 

To analyze the first research question, we present a descriptive assessment of UCI Bio 
93 freshmen’s MOOC performance compared to the other MOOC participants. To 
analyze the second and the third research questions, we construct a logistic regression 
model and a multinomial logistic regression model to investigate the factors that 
influence UCI Bio 93 students’ enrollment and final performance in the MOOC 
separately. 


Results 


There were differences in performance in the MOOC between the three groups 
examined: strong math UCI students, weak math UCI students and non-UCI MOOC 
participants. Typical of many MOOCs, the completion percentage of non-UCI 
participants was low, with 92% not completing. Of those that completed, approximately 
7% earned a Normal certificate and <2% earned Distinction (Figure 1). In marked 
contrast, the two groups of UCI students had a much higher percentage of completion 
and Distinction. For the strong math UCI students only 36% did not complete the 
course, while 37% earned a Normal certificate and 27% earned a Distinction certificate 
(Figure 1). The percentage of weak math UCI students (11=156) that did not complete the 
course (31%) was similar to the strong math group, but more earned a Distinction 
certification (39%) than a Normal certificate (30%) (Figure 1). 
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(n=27,105) (n=226) (n=156) 


Figure 1. MOOC completion and performance by group. Percentage of total students 
within each group who did not complete the course, completed the course and earned a 
Normal certificate, or completed the course and earned a Distinction certificate. Total 
number in each group indicated (n). 


Research Questions 2 and 3 focus on which UCI students enroll in and complete the 
MOOC. 

MOOC Enrollment 

Table 1 shows the odds ratios from the nested logistic regression models predicting UCI 
Bio 93 students enrolled in the Bio Prep MOOC, using data from the entire cohort of Bio 
93 students. As our hypothesis predicted, we found weak math students are more likely 
to enroll in the course. However, when we control for gender the effect of math is not 
significant. This indicates that women are more likely to enroll in the MOOC than men. 
There was no effect of ethnicity on enrollment. 
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Table 1 

Odds Ratios From Logistic Regression Models Predicting MOOC Enrollment 



Modell 

Enrollment 

Model2 

Enrollment 

Model3 

Enrollment 

Model3 

Enrollment 

Weak Math 

1.319* 

1.433** 

1.355* 

1.261 


(2.32) 

(2.66) 

(2.23) 

(1.63) 

SAT Reading 


1.000 

1.001 

1.001 



(0.37) 

(0.67) 

(0.50) 

SAT Writing 


1.001 

1.000 

1.001 



(0.60) 

(0.25) 

(0.47) 

Female 



1.604*** 

1.594*** 




(3.52) 

(3.47) 

Minority 




1.263 





(1.73) 

N 

1695 

1693 

1693 

1693 

R 2 

0.003 

0.004 

0.011 

0.013 


Notes. Exponentiated coefficients; t statistics in parentheses 
* p<0.05 ** p<0.01 *** p<0.001 


MOOC Completion 

We hypothesized that students with weak math SAT scores would be more likely to 
complete the MOOC with distinction, given the incentive provided by the university. As 
predicted, the probability of obtaining the Distinction certificate for students with low 
SAT math was about 1.66 times greater than the high SAT math students (Table 2, 
Model 1). This effect remains when the model controls for student preparation, gender 
and ethnicity. None of the other student characteristics examined significantly affected 
the prediction of earning a Normal certificate. 
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Table 2 


Multinomial Logistic Regression Predicting UCI Bio 93 Students’ MOOC Performance 


Modell Model2 Model3 Model4 

Performance Performance Performance Performance 

Distinction 





Weak Math 

1.660* 

1.720 

1.761 

1.995* 


(1.98) 

(1.76) 

(1.83) 

(2.12) 

Sat Reading 


1.002 

1.002 

1.003 



(0.88) 

(0.87) 

(1.04) 

Sat Writing 


0.999 

0.999 

0.998 



(-0.58) 

(-0.50) 

(-0.74) 

Female 



0.793 

0.796 




(-0.74) 

(-0.72) 

Minority 




0.680 





(-1.28) 

Normal 





Weak Math 

0.956 

0.667 

0.712 

0.680 


(-0.18) 

(-1.34) 

(-1.11) 

(-1.20) 

Sat Reading 


0.995 

0.995 

0.995 



(-1.79) 

(-1.81) 

(-1.86) 

Sat Writing 


0.999 

1.000 

1.000 



(-0.32) 

(-0.12) 

(-0.02) 

Female 



0.574 

0.578 




(-1.86) 

(-1.84) 

Minority 




1.158 





(0.50) 

N 

382 

382 

382 

382 

R 2 

0.007 

0.022 

0.026 

0.030 


Notes. Exponentiated coefficients; t statistics in parentheses 
* p<0.05 ** p<0.01 *** p<0.001 


Discussion 


Our results demonstrate that the percentage of UCI students completing the MOOC and 
earning Distinction was much higher than the non-UCI MOOC students. Female 
students were more likely to enroll than male students, and low math SAT students were 
more likely to earn Distinction. 

There would be little support for investing in development of college preparatory classes 
if the completion rates are similar to those reported for most MOOCs (~5%) (Ho et al., 
2014). A study at San Jose State University indicated that students in a MOOC who are 
also matriculated and the course counts for university credit have a higher completion 
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rate in three courses (29.8%, 50.0% and 54.3% respectively) than those who are not 
matriculated (17.6%, 11.9% and 48.7% respectively; Firmin et al., 2014). Our study 
demonstrates the completion rate for incoming UCI students in a biology prep MOOC 
can be boosted to over 60% even when there is no course or university credit involved. 
The exposure to knowledge and/or skills relevant to a first quarter class was sufficient to 
result in this increase since incoming 1st year students with the credentials to become 
biology majors completed at approximately the same rate as underprepared students 
who were given an explicit incentive. 

There is considerable concern that online education programs, while effective for some, 
may amplify rather than narrow educational and social divides (Bolt & Crawford, 2000). 
Some quasi-experimental studies show that online education reflects the same divide 
commonly observed in the brick-and-mortar settings. For example, Black and Hispanic 
students performed more poorly than White students in online courses (Newell, 2007). 
In an experimental study involving multiple sections of an economics course no 
significant difference was found between online and lecture courses among students 
with higher prior GPA; however, among students with lower GPA, online students 
scored significantly lower than face-to-face lecture students (Figlio, Rush, & Yin, 2010). 
In the San Jose State University study, the low course pass rates may be due to the 
target group being at-risk students (Firmin et al., 2014). 

In the present study we found that UCI incoming students with weak SAT math skills 
had a higher probability of completing the MOOC with Distinction than students with 
strong math skills. It is likely that this difference in completing the MOOC with 
Distinction is associated with the incentive policy enacted for incoming freshmen who 
did not meet the SAT math requirement for a biology major. This provided freshmen 
who obtained a Distinction certificate the opportunity to enter the biology major two 
quarters earlier than those who did not complete or earned only a Normal certificate. 
Importantly this incentive did not result in any costs for the students, the instructors, or 
the university. 

Our results also indicate that among UCI students, females were more likely to enroll in 
the MOOC than male students. This is interesting given previous research showing that 
the majority (5i%~87%) of MOOC population are males (Ho et al., 2014) and it is 
potentially important given the underrepresentation of women in many STEM fields 
(Beede et. al, 2011) and women’s greater likelihood to transfer out of STEM majors 
(Chen & Soldner, 2013). One possible interpretation, consistent with a recent study, is 
that female students are more motivated and adaptive to online education than male 
students in an educational setting (Xu & Jaggars, 2013). In contrast, unlike the Xu and 
Jaggars study, we did not find negative associations between ethnicity and enrollment 
or performance in the MOOC. 

Although previous literature shows that low performance students can be further 
disadvantaged by online education, our results suggest that a MOOC with no-cost 
incentives provides an additional learning opportunity for low-performance students. 
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Follow-up research will analyze the impact of the MOOC for students’ academic 
performance in the onsite Bio 93 course. Additional studies from other institutions and 
MOOCs will also be important in evaluating the effectiveness of MOOCs as preparatory 
courses for higher education. 
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